Troubleshooting: Deep learning
Issues
▪I see identity swaps during data acquisition
▪The nose and tail base points are not stable even when the contour of the subject is detected well
▪The results of two-subject tracking are not consistent when using the same video
▪The Data Preparation Report says “Failed” for some trials
▪A subject is labeled as Subject 1 in the first trial and Subject 2 in the second trial
▪The results of two-subject tracking differ depending on the graphics card (GPU) used
▪One of the subjects was not tracked at the start of the video
I see identity swaps during data acquisition
This is normal when using the Deep learning technique in two-subject tracking. The identity swaps are corrected during the Data Set Preparation process. During that process, the track segments are sorted and put together based on the visual appearance of the subjects. This procedure greatly reduces identity swaps. See Prepare the data in multi-subject trials
The nose and tail base points are not stable even when the contour of the subject is detected well
▪With one subject per arena: In the Detection Settings, click the Define button, then Automated.
▪With one or two subjects per arena: check that there is enough contrast between subjects and background.
▪Furthermore, objects in the arena may reduce the accuracy of the position of the nose point.
▪See also Deep learning: Requirements
The results of two-subject tracking are not consistent when using the same video
This occurs because the individual recognition algorithm is not entirely deterministic. Probability plays a role in determining whether an animal in a certain video frame is labeled as Subject 1 or Subject 2.
As a result, the statistics may differ between two trials, even when using the same video and tracking settings (Trial Control Settings, Arena Settings and Detection Settings).
▪Mirror labeling. The first thing you may notice is that, when you acquire two trials from the same video, one individual (say that with the tail marks) is labeled as Subject 1 in trial 1 (red dot; left), whilst as Subject 2 in trial 2 (green dot; right). Conversely, its partner is labeled as Subject 2 in trial 1, and as Subject 1 in trial 2. Thus, the second trial is a sort of mirror image of the first one.
If you want to keep consistency of labeling, for example to have the resident individual of a resident-intruder pair of mice always labeled as Subject 1, swap the subjects in the entire trial (see below).
▪Consistency of ID assignment. Even when the same individual is labeled as Subject 1 in both trials, there is a small chance that the results are not 100% consistent. Within short segments of the tracks, the identity labels may differ: for example, the marked mice is labeled as Subject 1 in trial 1, and as Subject 2 in trial 2.
Such segments last a few seconds and usually occur during situations of body contact. According to our tests carried out on videos provided by users, the probability that an individual at any time point in the video is labeled the same way in both trials, after correcting for the mirror labeling (see above), is 99.6% (range 83.5 to 99.9, n= 9). Low values occurred in videos where the animals were in contact most of the time, or videos of low resolution and with more noise.
In the Track Editor, review the video to evaluate the extent to which identity swaps occur. If that is not acceptable, swap the subjects where necessary, so that you get fully consistent results.
▪Consistency of general statistics. The non-deterministic mechanisms of the multi-subject tracker based on deep learning also cause the statistics to be not 100% repeatable when tracking multiple times from the same video. When analyzing the Total Distance moved in trials acquired from the same video, the coefficient of variation is 0.4 % (range 0.1 - 2.5). In practice, a video where an animal walks 100 meters could result in some 40 cm difference in distance moved between trials. Higher-quality videos produce variability of the results lower than 1% of the mean.
If this variability is not acceptable, open the Track Editor and swap the subjects where necessary.
See also
▪Prepare the data in multi-subject trials
A subject is labeled as Subject 1 in the first trial and Subject 2 in the second trial
You may want to keep the identity labels consistent. For example, to assign the label Subject 1 always to the resident individual, or the individual marked on the tail.
To make identity labeling consistent, swap the subjects for the entire trial.
1.Open the Track Editor (Acquisition > Edit Tracks).
2.Select the trial where you want to swap the subjects.
3.Click the header for the first subject (default: Subject 1) in the Sample List.
You may have replaced Subject 1 and Subject 2 by other role names in the Experiment Settings; see Subjects per arena).
4.Press Shift, and click the header of the second subject (default: Subject 2).
5.As a result, the entire sample list is highlighted in green. See also Select samples manually
6.Click the Swap subjects button.
See also
The results of two-subject tracking differ depending on the graphics card (GPU) used
The results of two-subject tracking could be slightly different based on which GPU was installed, everything else being equal (e.g., computer, arena settings, etc.
When the neural network model is not highly certain about the pose of the subject, small differences in handling of number with decimals could lead to noticeable differences, for example in the position of the nose at a certain sample time. Another reason could be that precision handling could differ between NVIDIA driver versions.
Tests carried out with supported GPUs show that the maximal difference in distance moved between GPUs is in the order of 0.1%.
One of the subjects was not tracked at the start of the video
When you open the Integrated Visualization, there is one track in each arena, while you expect two tracks. One of the two subjects apparently was not detected. This usually occurs in the first few seconds of the video.
In reality both subjects were detected, but their tracks now overlap. You can check this by de-selecting one of the subjects in the Show/Hide pane.
To prevent this from happening.
▪Create new Detection Settings and select the maximal sample rate, i.e. at least 25 samples per second, then re-do the trial. See Sample rate
▪Make sure that the two animals are released at the same time, or one shortly after the other (e.g. less than 30 seconds). The video should not contain a long segment with just one animal in the arena.